Software Vault: The Gold Collection

home *** CD-ROM | disk | FTP | other *** search

/ Software Vault: The Gold Collection / Software Vault - The Gold Collection (American Databankers) (1993).ISO / cdr52 / nguide.zip / NGUIDE.ART

Wrap

Text File | 1993-04-01 | 88KB | 1,601 lines

*Clipper in Network, tips, suggestions... Clipper in Network, tips, suggestions... alias Network Guide for Clipper programmer ──────────────────────────────── by Daniel Docekal Clipper application in network is something that should work on first moment of start if some guidelines are followed and some conditions fulfilled. If program you are writing is written with record and file locking and is taking care of everything that can happened in file sharing, there shouldn't be problem to run it and use it. Reality is often different, many problems can rise out from nowhere which will prevent your application from running or will give difficulties to application's run. What this article will try is to give helpful hand to those who are starting in networking with Clipper, but also for those who are swimming in problems with written Clipper application. I would like to ask anybody who wishes to contribute in this Network Clipper guideline to do so, then i can collect opinions and release them in new edition. NETWORK APPLICATION──┐ ─────────────────────┘ Network application in general must follow idea that it's not only one program running at the same time. Network application which is using ANY from resources of network must take good care of fact that in most cases it WILL SHARE them with other applications running at the same time and requiring access to the same resources. In most cases it's matter of programming languages to give tools for networking rather than programmers knowledge. Clipper is high level programming language in difference from assembler or basic of C language, therefore is expected that networking behavior will be covered with filling needs. NETWORK──┐ ─────────┘ Network is something most horrible and bad for every programmer, nightmare like environment with every problem hidden after not sufficient knowledge and not experienced people around. Believe or not, it's right definition of network for beginning. Users of network which are in most cases not even good equipped with knowledge of MS DOS are of course able to broke all rules which logically thinking programmer will have in mind when programming, therefore programmer must start thinking like average (or even, like under average) users. Difficulty of this mind change is paid in most cases by application which is created with best wishes and is ending SOMETIME in troubles. CLIPPER──┐ ─────────┘ Clipper is programming language offering enough to start writing network application. General guidelines are very simple: ┌╖ if database file has to be used at one time by MORE users (workstations ╘╝ will be used in all next text) then it must be opened as SHAREABLE What this is requiring is very simple, either USE..SHARED or SET EXCLUSIVE OFF before all USE which are not equipped with SHARED nor EXCLUSIVE keyword. │ Experienced users are probably falling away, but please don't rather │ skip those basic informations, later are coming more handy ones. What can be often source of problems: WS-A has file FILE opened as EXCLUSIVE and WS-B is trying top open it by USE (of any form). As far as WS-A has it exclusive there is no way how to access this file. WS-A has file opened as SHARED and WS-B is trying open it as EXCLUSIVE which cannot because may not gain EXCLUSIVE access to already SHARED file. WS-A has file opened with fopen() (or just by program like Norton Commander), no any other workstation is able to access file. USE command is NOT well made for network operations. it's requiring little effort from programmer because needs test of NETERR() function after USE if required operation was finished: │ USE ADRES SHARED NEW │ IF !NETERR() │ SET INDEX TO ADRESI1,ADRESI2,ADRESI3 │ ELSE │ // error situation │ ENDIF This is base fragment of opening a files in network. ANY application can be built with this method because it WILL work perfect even without network and what it also will - work inside of MS WINDOWs or any other multitasking software which also requires network methods of file access. Here we do have of course little problem, USE attempt is just made one time and not repeated again and again for case that condition will change and file will be ready for opening. That's of course trouble of programmer which must know in what conditions will program run. 24hours running mail server should again and again try to open files, but user program just shall only tell to user about problem and then exit. It's user's decision what to do, call supervisor or try it later. ┌╖ if any file has to be used at one time by MORE workstations then it must ╘╝ be opened also in SHARED mode. This must be based on FOPEN() function which IS allowing some MORE attributes of open 0 FO_READ Open for reading (default) 1 FO_WRITE Open for writing 2 FO_READWRITE Open for reading or writing addition which should be done: 16 FO_DENYALL Not allow ANYBODY else access 32 FO_DENYWRITE Write access of others denied 32+16 FO_DENYREAD Read access of others denied 64 FO_DENYNONE Full access of others permitted This extension of fopen() function will allow to control if two workstations can simultaneously access the same text file (for example). ┌╖ Any updates to database files are requiring record or file locking, ╘╝ unless database is opened in EXCLUSIVE mode. SHARED open database is suitable for READING records but not for writing. Basic conflict which can happened when two workstations will try to change database is that both of them will come into the very same record. It's something that may not be. Therefore RECORD LOCK is required before ANY operation which CHANGES value of FIELD in RECORD of database. Record lock is in general result of RLOCK() function which will TRY to lock record, but it must not be successful. In general record locking mechanism should follow: - try to lock record and if it goes fine just continue with operations - if request for locking failed there are several possibilities: - try it for INFINITE time - try it until user will stop it with STOP key - try it for DEFINED time (let say one minute) - try it for DEFINED number of attempts - try FIX reason of failed record lock request - stop immediate and tell user and not continue In general there are combinations usual given by: - it's trying some time and user can stop it by pressing STOP key Many from third part add-ons are offering replacement for rlock() function, therefore let them to do job. BBS systems and Compuserve will have for sure enough of those examples... The same as above written is valid about locking of WHOLE file. Locking of record is needed in case of operation which will change ONE record, locking of FILE is needed on operation which will be changing WHOLE FILE or in cases when is NEEDED to be sure that no other workstation can jump in middle of counting values and change some of them just second after they were counted. Function for file lock is FLOCK() and works exactly the same way as RLOCK(), only effect is on COMPLETE file. After FLOCK() issued is file still accessible for READING, but no-one can write to any from file's records. All rlock() or flock() operation tried by other workstation will be unsuccessful. Requirement for successful flock() is therefore: - no one has locked ANY from records in database - no one else has flocked() database General required are flock() or rlock() in functions/commands: rlock() flock() ───────────────────────────────── ──────────────────────────────────── := APPEND FROM DELETE (for one record) DELETE (for more records) RECALL (for one record) RECALL (for more records) REPLACE (for one record) REPLACE (for more records) @..GET (operating on field) UPDATE ON fieldput() dbeval() (changing records) fieldwblock() Considerably required is flock() in: SUM COPY TO LIST AVERAGE JOIN DISPLAY TOTAL LABEL dbeval() (counting values) REPORT LOCATE and CONTINUE SORT Those commands DO NOT change values in database at all, but are dependent FROM values in database. In some cases it's therefore good to think if it's normal and good that someone else will be able change values in database during some counting processes. Clipper is placing OWN method of record locking on file which is NOT\ compatible with methods used by dBASE, FOXBASE, "C" language or others. in general is this method based on locking one byte areas beyond any possible end of file. This method is picked up because of needed side effect, records are still READABLE even when lock is placed on them. In normal case, when record is locked by locking AREA of file in which record is, any locked area UNREADABLE on the same way as no-one can WRITE to them. Clipper record which is LOCKED _must_ be readable from other workstations, therefore locking is done by simulation of fake records beyond possible end of file. All of this has next bad side effect - any other software which wants to access Clipper files at the same time as Clipper will have to follow exactly the same method of record locking, otherwise it can only make big mess from Clipper database files. ┌╖ Several database operations are requiring EXCLUSIVE use of file and ╘╝ without this they will not operate at all. EXCLUSIVE use of database is giving application right to do ANYTHING application wants with database. It's therefore required for some Clipper operation. EXCLUSIVEly used database is not allowed even to open from other workstations. From this point of view is power of "locking" in this order: EXCLUSIVE used no one else can ACCESS database file, database MAY NOT be open by other workstations, otherwise is impossible to make USE...EXCLUSIVE flock() no one can CHANGE thing in database, rlock() or flock() from other workstations will fail, no one can EXCLUSIVE USE database rlock() no one can CHANGE record in database, rlock() or flock() from other workstations will fail, no one can EXCLUSIVE USE database SHARED used everyone can open database, use flock() or rlock(), but no can EXCLUSIVE open database Functions and commands for which EXCLUSIVE is required are: INDEX (*) PACK REINDEX ZAP dbcreateind() (*) dbreindex() INDEX/dbcreateind() are partially excluded from this list in case that on one is interested in multi-use of created indexes. INDEX/dbcreateind() are generating index files which are not available for other workstations by default, therefore EXCLUSIVE use of database is NOT required. But there is one important detail: Index file generated on non-exclusive (or non-flocked()) database is quite unstable in case someone just changed some values of keys in middle of creating index key, therefore as general requirements can be given: │║ Database _must_ be either file-locked or EXCLUSIVE used for │║ case of index generation. Only with this one can overcome possible │║ index destruction with later IE1210 (well know to most of us). ┌╖ Appending of new record is not requiring rlock(), but it's requiring ╘╝ test of success. APPEND/dbappend() are used for ADDING ONE new record at END of database. INSERT command which is present for example in FoxBase doesn't exist in Clipper and therefore problem with new records is eliminated only to appending at END of database. APPEND/dbappend() is by default trying to make one NEW record and then place rlock() at this record. If something fails (two workstations are doing dbappend() at the same time) dbappend() will not succeed and NETERR() will be again returning information about error. Good sequence therefore is: dbappend() while neterr() dbappend() enddo This sample of course is limited in never ending waiting for append of record, but what was told about flock() can be applied also in custom appending function. There are therefore few points about appending: - record is ALWAYS locked if dbappend()/APPEND _was_ successful, no need for additional RLOCK(), it's wasting of time - record is EMPTY after APPEND and MUST NOT be currently updated in database YET nor in index files There is no need for checking success of APPEND/dbappend() in cases of USE...EXCLUSIVE or flocked() databases, but considerable is fact that flock() active _before_ dbappend() is replaced with rlock() active _after_ dbappend() (see following comments). ┌╖ Clipper is limited in number of rlock()/flock() on ONE per DATABASE ╘╝ and per WORKSTATION (program). Limitation of Clipper is that there can be ONLY ONE rlock() or flock() per ONE database and workstation. One program can therefore place just one lock per one open file. It's somehow limitation for many applications, therefore many add-ons for Clipper are offering multiple locks per file (DBFSIX, NETLIB for example). Also important point is that ANY RLOCK() or FLOCK() is removing previously placed RLOCK() or FLOCK() in the same file, therefore programmer MUST be careful when placing locks in the same file. ┌╖ EVERY rlock() or flock() should have own dbunlock() or dbunlockall() ╘╝ function to assure proper unlocking and proper updating of data From moment of placing rlock() at record is very important to remember that ANY changes made to this record MUST NOT be visible for other workstations, even they have READ access to this record. They can see still OLD values and old KEY values in index! dbunlock() is only one which is REQUIRED to make changes visible. If this is not true in your network, then believe or not - your network or your DOS version has VERY important BUG, don't blame Clipper. On the same way as WHILE has proper ENDDO, IF has proper ENDIF, CASE proper ENDCASE it's very important to have for every RLOCK() proper DBUNLOCK(). There is of course possible to use DBUNLOCKALL(), but that's solution not very clean - in some cases it can unlock something that should stay YET locked. Thinking that NEXT rlock() (or flock()) will make dbunlock() automatically is of course right, but it has one side effect, one can miss easy proper lock-unlock combination in case of longer source files with more actions between flock() and later flock() again. It's of course easy to use it in way: while !oef() if rlock() ADRES->CITY := upper(ADRES->CITY) endif dbskip(1) enddo dbunlock() This sample is CLEAR on FIRST SIGHT, therefore nothing against this way of coding program, but in case that this source will become FEW pages of lines, it's much better use dbunlock() on right place. Time saving given by again and again using dbunlock() separately from rlock() are not very important because rlock() in above example will have to make unlock() of previous record anyway (with subsequent update actions). ┌╖ Using of dbcommit()/COMMIT can be sometime too much ╘╝ dbcommit()/COMMIT/dbcommitall() are in most cases very powerful. In processing od DATABASE files of Clipper are several levels of buffering (CACHING). First there is Clipper internal buffering of data in EMS or conventional memory (if available) which buffers complete records of database when possible, then there is DOS buffering (and Caching) which works between disk hardware and Clipper requests. It uses either buffers created by CONFIG.SYS command BUFFERS= or cache buffers created by CACHE software (SMARTDRV.SYS in standard). In network environment there is in some cases network buffering at workstation which is NOT active for Clipper files opened as SHAREABLE and which IS active for EXCLUSIVE open files. COMMIT/dbcommit() possibility of CLipper is related to those buffers: - DOS buffers are flushed TO DISK in versions 3.3+ - CLIPPER buffers are flushed TO DOS in versions under 3.3 - SERVER buffers are flushed to SERVER disk in Netware 3.11 with NetWare Shell of versions 3.22+ It's generally known and approved that dbunlock() is far enough to make visible record changes in all normal cases. Committing data is in most cases even not very good idea because it can make additional unwanted traffic inside of server between server CACHE memory and physical disk. Warning anyway, there ARE revisions of DOS which are affecting file handling functions and which HAVE significant problems with file and record related function (not even locking) when running under Networks. One sample can be TANDON DOS version 3.3 which is exception from case that dbunlock() is enough for making updates visible, dbcommit() IS required in this case to SEE dbunlock() activated. ┌╖ DOS based networks are REQUIRING use of SHARE.EXE ╘╝ Rule about DOS based network is that ANYTHING that is coming to be SERVER and will have to share files for other workstation MUST have loaded SHARE.EXE program BEFORE any other network programs OR in some SPECIAL order when it's specified by documentation for network. What is coming to be problem is that SHARE.EXE by default is ready only for SINGLE user machine with limited number of files opened. MultiUser SHARE.EXE requires some changes in parameters. SHARE.EXE has TWO parameters and common knowledge of them is not very good. /F:n is used to specifying SIZE OF FILE NAME POOL which SHARE.EXE will allocate. Standard size of this pool is only 2048 bytes and therefore very small. ONE opened file from server will in general require about 20bytes for name + 20 bytes (exactly 11bytes of information and full size of path+name) for informations, and that means that 2048 is space for 50 files. Common Clipper programs are easily opening this number of files PER PROGRAM. Therefore take number of workstation accessing your server, count let say 60 files to open from every workstation and result will be <n>*60*40. Be careful in thinking, because MEMORY WILL BE ALLOCATED from CONVENTIONAL memory of server (in most cases) and therefore CAN limit significantly any other operations. /L:n is used for specifying SIZE OF LOCK POOL which SHARE.EXE will allocate. Standard size of this POOL is allowing 20 locks. That's also not good enough because Clipper program CAN make the same number of locks as is number of files it's opening. Therefore get number of workstations multiplied by number of files (<n>*60 in sample case) and that's /L:n which should be used. This parameter is NOT increasing memory usage so much, therefore don't worry about it. ┌╖ DOS based network are requiring .EXE and .OVL files marked as READ ONLY ╘╝ DOS based network (and may be that some others also) are requiring that .EXE and .OVL files from Clipper program will be marked as READ ONLY to allow more than one workstation run the same file at the same time. It's problem of DOS default method of opening non-Read-only file which is making impossible to open it twice by default. Therefore just don't forget to mark them with ATTRIB as +R. ┌╖ WORKSTATION is NOT requiring SHARE.EXE ╘╝ ANY workstation running in ANY network doesn't need SHARE.EXE, don't worry about clever recommendations to load SHARE.EXE on all workstations of your network. It's wrong. It will only take up your memory (which is critical in many cases) and will do nothing, or worse will conflict with your programs and make problems. From this rule is only ONE exception. Microsoft DOS 4.01 running workstations with HARD DISK bigger than 32MB _are_ _requiring_ SHARE.EXE to be loaded. IF it's not loaded computer will run smoothly until disk is started to be allocated in area over 32MB and some stupid program using old way of file handling (like SideKick for example) will be started. After it harddisk of workstation is overwritten... ┌┐ Computers or workstations running MULTIUSER software and planning └┘ running multiple Clipper programs over the same data NEED SHARE.EXE Anything running DesqView, MultiDos, MS Windows or anything else which is providing multi tasking capabilities should load SHARE.EXE before start of multiuser software otherwise lower copy of DOS will not be able to handle file/record locking and shared accessing. In general multi task environment SHOULD BE TAKEN AS EQUAL TO SIMPLE NETWORK, therefore anything written before or after this statement can be active... ┌╖ Novell Netware SHAREABLE flag is DANGEROUS, DO NOT try mark your files ╘╝ as SHAREABLE (FLAG +S) SHAREABLE flag of Novell Netware files is something that MUST be avoided to use for Clipper data files. SHAREABLE flag has for Novell Server only one meaning that it should ignore any file/record locking on given file because applications using file will maintain OWN method of locking and checking. As far as CLipper is still using standard method of locking, only far beyond end of file it means that any locks can went well wrong when file is marked Shareable. Please try understand this statement as copied out from official Novell statement which can be made available and therefore please understand this as end of myth of Shareable attribute. There is no reason to mark Clipper database file on any special way in any known network, especially not in Novell Netware. ┌╖ Be sure that Clipper application users have enough rights in directories ╘╝ where CLipper program will be working. Often problem is that there is not enough rights for Clipper .EXE file to run and to access his data. In Clipper it's not very difficult, just give ALL rights in directory where .EXE is running, in TEMPorary directory and in all directories where database related files are located. I didn't yet made experiments with giving less rights, but generally it shouldn't be a problem because CLipper application will need anyway separate direc- tories for program and data and giving rights in secured network will not be very big problem. There is also little bug in OLDER versions of Netware. FILES in Netware are ALWAYS created WITH OWNER. It's good idea to create ALL files AS SUPERVISOR because this one will NEVER be deleted. IF is installed limiting of space by users it can be that file create by let say user named FRED will give strange errors for one simple reason. User FRED is already deleted from server and therefore space available checking mechanism is unable to determine if it can append new records to file and will give "Disk Full" messages even when there are MEGABYTES of free space. Just change owner of files to SUPERVISOR or somebody who is existing in network and problem will be solved. ┌╖ Don't forget for TEMP drives! ╘╝ TEMP drive specified for application in SET CLIPPER can be source of problems when someone will forget to create this directory or don't give rights for users in this directory. Clipper application WILL create sooner or later some swap and temporary files in there and if it will be unable to locate directory? It will just jump out of program without possibility to catch this error! It's GOOD idea to collect ALL temporary files in separate directory called for example TEMP (or $TEMP$) because they are garbage in meaning of network. If workstation HAS harddisk or RAMDISK is good to redirect those temporary files on them. It can save LOT of megabytes transferred through network and also therefore can speed up your application. ┌╖ What about marking some directories as PURGE in 3.11 Netware ╘╝ Since Novell Netware 3.11 is Novell server SAVING _all_ deleted files for undeletion. What it means is usage of disk space on server (he will purge them automatically when needed) and also time required for those and later for PURGE operation. It's nice to be able undelete by accident deleted file two weeks ago, but in some cases it's good to use PURGE flag for directories (command is FLAGDIR <dir_name> P). Directories marked with this Purge flag will not try to keep files and all deleted will be automatically immediately purge. Good candidate for FLAGDIR P command is TEMP directory in ALL cases, otherwise Novell Netware server will have after week of working with your Clipper program thousands of files stored in salvageable state and for nothing. ┌╖ Quest for file handles ╘╝ Coming into troubles with file handles is very easy. General rules will be described just step by step to help cover this kind of problems. First of all, Clipper is VERY hungry for file handles (all other xBase derivates are anyway also because of using the same technique of file management), one therefore must be able calculate needs of his own program for file handles: - one file handle for .EXE file *ALWAYS* - one file handle for every .OBL file *WHEN USED* - two file handles for swapping files *ALWAYS* this can be even more when one allows more, SET CLIPPER=DYNF:<n> is used for this - one file handle per .DBF file *ALWAYS* - one file handle per .DBT .FPT file *WHEN USED* - one file handle per every .NTX .IDX .NDX *WHEN USED* - one file handle per every .CDX *WHEN USED* - one file handle per every fopen(), fcreate(), report, label, set alternate, set print to, set device to, append from, copy to, copy file, copy structure, create, create from, save, restore, sort, type, update *WHEN USED* - one or more handles when CLD.EXE or CLD.LIB *WHEN USED* is used (.PRG is open at least). Most file hungry moment from all those is fact of .DBF needs another file handle for any memo fields (.DBT file) and then for EVERY index related to one .DBF is required additional file handle for .NTX (or .IDX or .NDX) file. It's most limiting issue in using file handles and it can be solved only by two ways - one is requiring structural changes of applications, second one using of SIX RDD which allows using of .CDX file which can include ALL (and even more) .NTX in one file handle... (our main application which is user part of FAX/TELEX/MAIL program was using 45 file handles with DBFNTX driver, now it's using 26 file handles with DBFSIX driver). After counting this "BIG" number of files which application is using is good to write this number down and start setting up system to allow use of them. First place where one meets FILE HANDLEs specification is CONFIG.SYS, command FILES=<n> is giving maximum number of file handles which may be used in computer (workstation) and is by default ONLY 20, therefore for Clipper needs to be a bit more. Every file handle from FILES= will of course grab additional memory, therefore is not good to try allocate TOO much of them. But for sure it MUST be _more_ than "BIG" number from Clipper application. In general "BIG"+5 is good for <n>, but it's true for standard MS DOS without TSR programs (Norton Guide engine NG.EXE is for example using one file handle for NG.INI file!). 4DOS replacement of COMMAND.COM can keep open additional file handles for swapping files, therefore "BIG" and <n> should be more far from each other or "BIG" should be bigger than real application need. One thing is for sure anyway, LOCAL running application WILL NEVER be able open more than <n> files (allocated in CONFIG.SYS). Little warning, QEMM has nice utility called FILES.COM which CAN create additional file handles in High Memory and save conventional memory for application. In this case is FILES=8 most sufficient in CONFIG.SYS and later FILES <n>-8 in AUTOEXEC.BAT will do additional allocation of required. But warning, FILES utility from QEMM is _not_ compatible with file handles used in DR DOS 6.0 and even if it will not complain it will NOT give required file handles and Clipper application will be magically refusing to work even if all looks fine. SECOND place where is requirement to specify file handles in command line for your .EXE (or in SET CLIPPER or in linker burn-in script). Clipper written application is using Microsoft like run engine which by default is getting only 20 file handles from DOS under. Call for DOS service to getting additional file handles is required for opening more than 20 of them. Clipper is not so clever to just call at beginning for maximum (255) and is requiring programmer (user) to give request for this number. SET CLIPPER=F<x> or //F<x> for .EXE run is therefore REQUIRED to get more than 20 file handles available. GOOD TIP. If your linker is allowing burn-in of Clipper environment, just burn in F255 and forget about this nasty and stupid parameter. It will not give any harm or extra memory used (yes, ONE byte per file handle rounded to 16bytes boundary which is harmless in Clipper memory usage) and no longer thinking about which F<x> parameter is active now. Until now we were talking about SINGLE user/task environment on single PC. Different situation is coming in network and that's what we should discuss especially. Novell Netware is most often used network, therefore we can start there. First, Novell Netware 2.x servers are GENERATED with FIXED number of available file handles and MAXIMUM possible is only 1000 (that's sufficient for 16 workstations opening 60 files for example!). Because using of more available file handles on servers requires more memory in server some technicians generating 2.x servers were afraid of putting more MB of memory and generated servers with 200 handles available (for example). That's something that can break your plans to run application from CLipper in give server immediately. Therefore make always check for how much of files is server generated. Ask supervisor of network for look into SYSCON utility, Statistics submenu and General statistics. All number are there, even number talking about current and peak number of opened files. From those three numbers is possible start thinking about ability to run Clipper program. Example: Maximum Peak Current 600 560 320 This will be not very good place for running CLipper program, unless this program will be the one running at most workstations at the same time which will slow down normal workload of files. 560 in peak is VERY near of configured MAXIMUM and that's a sign of need to change maximum value. Problem. Novell 2.x server is GENERATED from about 30 floppies and then LOADED into harddisk by special loader, ANY changes of parameters of server's operating system are requiring NEW generation based on OLD definitions. In most cases 2.x server is impossible to generate again because original supporter has not floppies anymore or doesn't exist. Novell Netware 3.11 server is NOT limited in number of files for opening in generation or maximum given by exact number. Only available memory of server is limitation of number of files to open (and speed of course), therefore if your application is going to be running on 3.11 Network and server, no problem at all. Every workstations in Novell Netware network will have available 40 file handles by default WITHOUT giving any specification. If there is requirement for more it's needed to place request for it in NET.CFG (or SHELL.CFG) file. SHELL.CFG is _old_ used name which is replaced by NET.CFG in last days. Command to place in there is FILE HANDLES=<m> and it will assure ability to open maximum of <m> files FROM NETWORK. IMPORTANT NOTE. Files allocated in CONFIG.SYS by FILES=<n> and files allocated from network by FILE HANDLES=<m> are INDEPENDENT from each other. <n>+<m> MAY NOT exceed 254 and in old versions of Netware Shell (NETX) it will NOT give any warning when this is happening and will allocate just default (40 or even less). New netware Shell is giving warning and refusing to load when <n>+<m> is over 254. <n>+<m> is also NUMBER OF FILES AVAILABLE for OPEN _AT_ workstation, but <n> is talking about LOCAL FILES and <m> about NETWORK FILES (files from network). This fact can also affect CLipper application with F<x> setting by the way (that's why F255 is most nice solution also). From this comment is also coming out one other warning, NUL device which is using file handle when opened more times is using ALWAYS local file handle and not Netware file handle when used for testing number of file handles available for opening. Non-Netware networks can have similar limitations of files available to open in server. Maybe someone else is able to elaborate on this for Lan Manager networks, but we do not support them nor use them (for enormous slowness) and it's hard to say. DOS based network servers are using different ways of overcoming problem that MS DOS allows maximum of 255 file handles (file handle is 8bit number only) and therefore is ALWAYS matter of configuration of server HOW much of files it will allow to open simultaneously. Some from DOS based networks are even limited to 255 files maximum (old versions of Lantastic were for example) or have significant difficulties when trying to successfully and reliable work with more than 255 files. Always check with network supervisor for setting and count them in Clipper application installation. ┌╖ Versions of network support software or network operating systems ╘╝ There was already article devoted to Novell Netware 3.11 patches and their importance in correct working network and also article about Netware Shell releases and their troubles. Both are available in Clipper BBS Magazine number 14 and are also uploaded to Compuserve's forum CLIPPER and DBADVISOR as text files free to download. In general, IF application is not running properly be sure to take a look and check if networking software is latest version or it's patched with latest available patches. In world doesn't exist software WITHOUT bugs and shortcuts and therefore patches are something very usual in serious companies (NANTUCKET was exception because waiting for 5.01 patch of CLipper 5.0x was very, very long). Example can be taken on Novell Netware 3.11 operating system which has few very important bugs related immediately with Clipper (and any other database applications) which can lead in major problems. On the same way it's VERY important to use latest Shell programs (Shell programs are those who allow workstation to access network files and services - those like IPX/NETX or ODI/NETX in Novell Netware, NET in Lantastic and so on) are of latest versions also. MANY from problems are related to old version of Shell program which has well known and already fixed bugs. In Novell Netware it's matter of having just latest 3.26 version of NETX because before this one is one important problem related to VISIBILITY of updates in network... ┌╖ Memory problems ╘╝ We all know that CLipper is very tight with memory usage. RTLINKed application is maybe better to not describe so much because standard RTLINK distributed with Clipper is not very good in handling big programs with C, ASM and CLIPPER combination. Fortunately we do have our BLINKERs, WARPLINKs which are better in memory handling, but still it's not optimal and it's caused by Clipper philosophy. Clipper us using VERY large runtime engine which is present in every .EXE created even if it's one line program. It makes some base of memory usage which CANNOT be reduced very much. All additional CLipper code and other libraries are sooner or later coming to create monstrous applications with .EXE file over 1MB in size and load sizes over 350KB (load size is size reported by linker when finished work on bunch of .OBJ and .LIB files). Those monstrous programs are requiring about 350KB of just memory loaded with programs and at least another 100KB of memory used for program to have it running. It comes in 450KB of memory needed for POSSIBILITY to run program. That's not so big problem in SINGLE user computer because typical free memory in there can be some 550KB in older DOS versions and some 600KB in DOS 5.0 with DOS loaded HIGH. Where problem is coming is network because Shell drivers are requiring some memory for work also. Free memory for Clipper program then can end in between 50KB till 90KB less than in non-network case. And that can become critical in monstrous megabyte applications. This part of our long article will therefore try to give little light and magic into memory issue. Don't expect too much, it's just list of practical experiences from own life. - reduce size and load size of Clipper application ■ Do not use PLL for living versions of your applications. It's saving of course size on your disk but .PLL is requiring some additional impact on load and run size of application ■ Do not use incremental linked application in living environment. Sounds silly, but that's often happening that application linked with INCREMENTAL ON (we are NOT talking about RTLINK where incremental link is now working at all) is by mistake used. Incremental linking is making .EXE file bigger by padding all functions by additional empty space for later replacing relink. It requires therefore more memory for running of program (much more). ■ Do not use application linked with debugger or debugging code for living environment. /B switch of CLIPPER.EXE will generate object file which will force linkers to make .EXE file which is bigger and also requires more memory. Be sure that NO ONE from .OBJ files in your link script (or in your libraries) is compiled with /B because it's enough to have one to have included all debug informations. It's good (and needed for testing), but unwanted for living environment. ■ Reduce size of your .OBJ code by excluding line number informations for living environment. /L switch of CLIPPER.EXE is required for doing so. Side effect of this is of course that all ERROR reports will have line number ZERO, but living application shouldn't produce so much errors anyway. Line number is TWO BYTES in .OBJ for EVERY line of your SOURCE included in source and that can mean significant saving of space! ■ In debugging cycle link CLD.LIB into your application instead of using separated CLD.EXE. Save of memory is NOT SO BIG, but can make significant difference in ability of debugging own code. Don't forget in this case that CLD.LIB _must_ be included in linking script as FILE CLD.LIB and not as SEARCH CLD (or ALLOCATE CLD, LIB CLD) because it's nice present from Nantucket. But then, DO NOT forget to REMOVE CLD.LIB from link script for living environment application, otherwise anyway inactive CLD.LIB will take additional memory. ■ Don't think that just specifying list of libraries and object modules for your linker will take care of proper and optimal overlaying. Most from libraries are combination of CLipper and C/ASM code and should be overlaid explicitly by specification. Most from CLIPPER.LIB can be also forced to overlay and all this can save lot of memory. Don't forget in this case of course that invoking static overlay manager will require some additional memory (RTLINK about 18KB) and therefore effect from overlaid code must be bigger than impact of overlay manager present in .EXE code. Some examples of scripts for BLINKER 1.51 are here: TELEPATHY - communication library ┌───────────────────────────────────────────────┐ │ search TPROOT │ │ BEGINAREA │ │ allocate TPOVL │ │ ENDAREA │ └───────────────────────────────────────────────┘ NOVLIB - Novell Netware library ┌───────────────────────────────────────────────┐ │ BEGINAREA │ │ allocate NLEXTEND, NLMAIN │ │ ENDAREA │ └───────────────────────────────────────────────┘ EXTEND - Part of Clipper libraries ┌───────────────────────────────────────────────┐ │ BEGINAREA │ │ ALLOC extend │ │ ENDAREA │ └───────────────────────────────────────────────┘ CLIPPER - Main Clipper library ┌─────────────────────────────────────────────────────────────────┐ │ SEARCH CLIPPER │ │ # tbrowse are forced in ROOT otherwise it's too slow │ │ MOD tbrowse0, tbrowse1, linelen │ │ │ │ # needed for NETLIB library │ │ MOD txopen, eve, filesys │ │ # those marked with ## is possible to overlay too but │ │ # main issue can be slowness of system in some cases │ │ BEGINAREA │ │ MOD accept, acopy, adel │ │ ## MOD aeval │ │ MOD ains, appinit, atail │ │ ## MOD box , color │ │ MOD cmem │ │ ## MOD date │ │ MOD dbcmd0, dbcmd1, dbcmd2, dbcmd3, dbcmd4, dbcmd5 │ │ MOD dbcreate │ │ ## MOD dbf0, dbf1, dbfdyn │ │ MOD dbjunct, dbnubs, dbstruct, delimdyn, dlm0, dlm1 │ │ ## MOD dtx0, dtx1, dtxdyn │ │ MOD diskio , dynina │ │ ## MOD errorsys │ │ MOD errsys0, errsys1 │ │ ## MOD event, extend, fget, field │ │ MOD getenv, gets0, gets1, gets2, gx, joinlist, lupdate │ │ ## MOD maxrow │ │ MOD memory, mrelease, msave, net, nmsghdr │ │ MOD oldbox, oldclear, philes, run, saverest, scroll │ │ MOD sdf0, sdf1, sdfdyn, send, seq │ │ ## MOD set, setcurs, sortbloc │ │ MOD sortof, squawk, startsym │ │ ## MOD symsys │ │ MOD tb │ │ ## MOD txopen ## NETLIB needs it in root │ │ MOD vall │ │ ## MOD vblock, vdb │ │ MOD vdbg, version │ │ ## MOD vmacro, vnone, vops, vpict, vterm, workarea, wrt2err │ │ MOD xmacro │ │ │ │ MOD _afields, _appini, _atpromp, _century │ │ MOD _dbcopy, _dbdelim, _dbflist, _dbghelp │ │ MOD _dbginsp, _dbgmenu, _dbjoin, _dblist │ │ MOD _dblocat, _dbsdf, _dbsort, _dbstrux │ │ MOD _dbtotal, _dbupdat, _fledit │ │ ## MOD _getmsg, _getsys │ │ MOD _helpkey, _input, _readvar, │ │ ## MOD _savescr, _setfunc │ │ MOD _setta, _text, _wait │ │ │ │ MOD memoedit, memotrain, memoread, memowrit │ │ MOD memoline, mlcount, mlpos │ │ │ │ MOD is, examplec, hardcr, directry, copyfile │ │ MOD typefile, diskspac, achoice, osdate │ │ MOD examplea, initexit, strtran │ │ │ │ ENDAREA │ │ │ │ SEARCH terminal │ │ # i do not use DBFNTX anymore, therefore i removed it out │ │ # SEARCH dbfntx │ └─────────────────────────────────────────────────────────────────┘ DBFSIX - RDD replacement of DBFNTX ┌─────────────────────────────────────────────────────────────────┐ │ FILE DBFSIX.OBJ │ │ #FILE DBT.OBJ │ │ FILE DBCREATE.OBJ │ │ │ │ # Speeds up index creations when marked out │ │ #MODULE SXCREATE │ │ # using standard overlay link from Successware │ │ @\dbfsix\dbfsix.lnk │ │ │ │ BEGINAREA │ │ ALLOCATE DBFSIX │ │ ENDAREA │ └─────────────────────────────────────────────────────────────────┘ NETLIB - Networkin library ┌────────────────────────────────────────────────────────────────┐ │ MOD A0FIX, C1LOCK, A0BREAK │ │ │ │ BEGINAREA │ │ allocate NL150, HORIZ50 │ │ ENDAREA │ └────────────────────────────────────────────────────────────────┘ Dr.Switch ASE - swapping and making program TST is ALL in ROOT ┌───────────────────────────────────────────────────────────────┐ │ MODULE ASE │ │ SEARCH ASE │ └───────────────────────────────────────────────────────────────┘ NANFORUM TOOLKIT - best public domain library ┌───────────────────────────────────────────────────────────────┐ │ BEGINAREA │ │ ALLOC nanfor │ │ ENDAREA │ └───────────────────────────────────────────────────────────────┘ BLINKER general script for LIVING environment application ┌───────────────────────────────────────────────────────────────┐ │ # use a fixed overlay area instead of the free pool │ │ BLINKER OVERLAY FIXED │ │ # │ │ # allocate 60K to the overlay pool │ │ BLINKER OVERLAY OPSIZE 60 │ │ # │ │ # full link only │ │ BLINKER INCREMENTAL OFF │ │ # │ │ # call stack size and fixes stack size │ │ BLINKER PROCEDURE DEPTH 55 │ │ STACK 5120 │ │ # │ │ # overlay in EMS aare disable because of using EMS in others│ │ BLINKER OVERLAY PAGEFRAME OFF │ │ # │ │ # and use upper memory blocks for overlay pool to save memor│ │ BLINKER OVERLAY UMB ON │ │ # │ │ # Burn in environment │ │ BLINKER EXECUTABLE CLIPPER F255;CGACURS;SWAPPATH:'TEMP\' │ │ # │ │ # allow users override environment with SET CLIPPER │ │ BLINKER ENVIRONMENT OVERRIDE │ │ # │ │ # set different than SET BLINKER │ │ BLINKER ENVIRONMENT NAME APPSET │ │ # │ │ # don't delete .EXE file when error │ │ BLINKER EXECUTABLE NODELETE │ │ # │ │ # burn-in serial numbers │ │ BLINKER EXECUTABLE SERIAL (c)NETCONSULT 1992, (DD) │ │ # │ │ # display list of all duplicates │ │ BLINKER MESSAGE DUPLICATES │ │ # │ │ # don't blink, it's bothering me │ │ BLINKER MESSAGE NOBLINK │ │ # │ │ # don't search for default libraries! │ │ NODEFLIB │ └───────────────────────────────────────────────────────────────┘ BLINKER general script for TESTING of application ┌───────────────────────────────────────────────────────────────┐ │ # uncomment this when needed listing on the screen │ │ #VERBOSE │ │ # │ │ # uncomment this when needed debugging/profiling │ │ #DEBUG │ │ # │ │ # uncomment this for creating a MAP file │ │ #MAP │ │ # │ │ # use a fixed overlay area instead of the free pool │ │ BLINKER OVERLAY FIXED │ │ # │ │ # allocate 60K to the overlay pool │ │ BLINKER OVERLAY OPSIZE 60 │ │ # │ │ # incremental link as standard │ │ BLINKER INCREMENTAL ON │ │ # │ │ # call stack size and fixes stack size │ │ BLINKER PROCEDURE DEPTH 55 │ │ STACK 5120 │ │ # │ │ # overlay in EMS aare disable because of using EMS in others│ │ BLINKER OVERLAY PAGEFRAME OFF │ │ # │ │ # and use upper memory blocks for overlay pool to save memor│ │ BLINKER OVERLAY UMB ON │ │ # │ │ # Burn in environment │ │ BLINKER EXECUTABLE CLIPPER F60;CGACURS;INFO │ │ # │ │ # allow users override environment with SET CLIPPER │ │ BLINKER ENVIRONMENT OVERRIDE │ │ # │ │ # set different than SET BLINKER │ │ BLINKER ENVIRONMENT NAME APPSET │ │ # │ │ # don't delete .EXE file when error │ │ BLINKER EXECUTABLE NODELETE │ │ # │ │ # burn-in serial numbers │ │ BLINKER EXECUTABLE SERIAL (C) NETCONSULT 1992, (DD)ß │ │ # │ │ # NO display list of all duplicates │ │ #BLINKER MESSAGE DUPLICATES │ │ # │ │ # don't blink, it's bothering me │ │ BLINKER MESSAGE NOBLINK │ │ # │ │ # don't search for default libraries! │ │ NODEFLIB │ └───────────────────────────────────────────────────────────────┘ ■ Using of Clipper code is better for memory than C/ASM written code in cases when SPEED is not very important. Dynamic overlaying of Clipper code in ALL cases is giving better memory usage with side effect on speed of course. Most from C/ASM code is very fast, small but overlaid in not so good way as Clipper. But all this is question of used linker of course. ■ do not allocate unneeded variables, PRIVATE or PUBLIC variables. Make variables which are really needed and do them as LOCAL or STATIC. All others can have impact on memory when used. It's saving of bytes, but who knows maybe some of them will be needed... ■ When using C/ASM written code try to use Clipper VMM system as less as possible because it requires additional allocations of memory which are impossible to release until is specifically told by your C/ASM program. - make as much as possible of free memory for Clipper program. ■ Use MS DOS 5.0 (or DR DOS 6.0 but there are known compatibility problems of DR DOS with MS DOS which can affect any application) with HIMEM.SYS and DOS=HIGH or with something providing UMB and load everything high that can be loaded high. ■ Don't make unneeded FILES=, BUFFERS=, STACKS=, FCBS= allocations in CONFIG.SYS, for network environment can be told FILES= can be 8 in Novell Netware because FILE HANDLES= in NET.CFG are more important BUFFERS= can be 8 in any Network because buffers will not be used in networking access and therefore are relevant. Every buffer needs own memory for allocation. BUFFERS are anyway good in multiplications of at least four. If BUFFERS= is possible to move in EMS, HIGH memory do so (HIBUFFERS, BUFFERS=/X ...) STACKS=0,0 always. Nobody needs additional STACKs from MS DOS and some resident programs are requiring this anyway. Can save some space FCBS=at absolute possible minimum (version of DOS dependent) ■ Use replacements of COMMAND.COM like 4DOS (or NDOS) which are using only about 3KB of memory for COMMAND.COM and loading rest in EMS,XMS or just leaving as overlay on disk. I'm using 4DOS already several years and it's fully compatible COMMAND.COM replacement which cannot cause any troubles to any programs. ■ Do not use relevant device drivers like ANSI.SYS in system where nothing with ANSI will be used, COUNTRY.SYS when nobody needs COUNTRY support, EGA.SYS, DRIVER.SYS or so. Every device driver needs memory which can be handy for Clipper program. If those drivers are needed, use HIGHDEVICE, DEVICEHIGH or so for loading them to available high memory to save conventional memory for Clipper program. ■ Do not load relevant TSR programs like FASTOPEN in network, caching software in network, DOSKEY when user will not be using DOS command line at all because will walk whole day in menu. All neccesary TSR programs is again good to load high (when possible) ■ Don't load relevant parts of network drivers. When nobody needs NETBIOS emulation don't load it in Novell network. When nobody needs SPX and Diagnostic communication don't load those parts of IPXODI (when using DOS ODI drivers) ■ Use DOS ODI drivers instead of IPX/NETX in Novell Netware because it's offering less memory usage and can even be reduced by excluding mentioned SPX/Diagnostic part. ■ Use memory managers software like QEMM, QEMM386, QRAM, 386MAX, BLUEMAX, NETROOM, HIDOS with possibility of creating up to 300KB of HIGH memory for loading of ALL used TSR programs, device drivers, files, buffers, fcbss. With programs like this can complete workstation in DOS 5.0 really end in nearly 640KB free after loading all programs including network drivers. ■ Use memory managers providing EMS memory (QEMM, EMM386, 386MAX, NETROOM and others) which can then be used by Clipper paging and overlaying system. It can help in speed increase of CLipper application and also in some memory savings because of more accessible memory. ┌╖ That stupid hanging SHIFT key no my keyboard! ╘╝ In most from the networks is happening that SHIFT key is sticking ON even it's not really pressed. It's not problem of hardware, it's problem of software. Redirection of keyboard interrupt routines can become too long that it can start loosing status informations. In Networks it's common problem. There are PATCH programs which will solve this and is therefore highly recommended to get them before complaining about this Clipper programmers which are messing my keyboard. Look for KBFIX, KBDFIX, KEYBFIX, SHIFTFIX, INTFIX INTxxFIX and similar words when scanning for this patch. It's for sure available in NOVLIB... ┌╖ Data integrity problems are in most cases problem of hardware ╘╝ Hardware CAN cause terrible problems which looks like software bug Believe or not, there were MANY cases of software-like problems which were caused by hardware problems, therefore following is a list to check when problems exist but all other possibilities are already checked out. - faulty network card in server or in workstation ANY faulty network card which is successfully working for common DOS tasks can fail under heavy Clipper load. It can fail most easy because of amount of transferred data which Clipper applications normally does. Any network MUST have possibility to CHECK network cards for transferring errors. In normal cases must LAN have 100% error free transfer between workstation and server. If it's not true for some from network cards then replace them for sure. - memory problems of workstation or server Because of heavy user of memory by Clipper application can easily happen that faulty memory chip is not failing under normal conditions, but only with Clipper programs (using for example EMS memory somewhere at end of memory). If Parity error is not generated wrong information can be obtained from memory and then used. Weak memory chips (SIMMs or SIPPs) are often successfully working with programs like MS DOS and are even able to pass initial memory test of your computer (anyway this is not real test as it should be), but are visible under programs really checking memory (like CHECKIT). Sometime computer MUST run longer time to get overheated a bit and then test can be made to see problem. - conflict in memory or in hardware Believe me that technicians installing network cards on IRQ2 are still between us (IRQ2 is used in EVERY AT computer for all IRQs 8 till 15 and therefore is NOT free for network card). Computer or server with like this installed card WILL work. Will work slower and will have reliability problem in some cases because at least HARDDISK controller will be located between IRQ8 till IRQ15 and therefore IRQ2 will be shared by TWO devices. ANY conflict of hardware or software (like non deallocated ROM for QEMM which must be) can be source of problems. If program is running OK on _one_ computer and WRONG on _other_ computer, then be sure it's not problem of program (in 99% cases) but problem of computer. - conflict with resident software There are MANY resident program which can cause problems for not only Clipper application. One example should be USER.COM from LANSIGHT software. When this resident program is loaded, Clipper applications acts like mad program, crashing in unpredictable moment or smashing your database files. Therefore be always careful what TSR is loaded. Test problematic program also on PURE MS DOS loaded computer. - bad sector on your server There MAY be problem for example in Novell Netware server that sector which is BAD is _not_ marked as bad and is used again and again. It's hard to understand how this can be happening, but i'm suspecting that HotFix on some servers can be full (or off) and server can still run without someone notice it. In this case your database files will get corrupted on network, but will run fine in local disk or on other volume. What can also make problems is: ■ too old version of disk driver ■ IDE (ISA) disk with bad driver or wrong setup in CMOS ■ disk which has ON-BOARD cache memory (like some Seagates) and this memory is not switched OFF. In server there MAY NOT be onboard cache memory on harddisk! ■ NLM which is responsible for disk/io operations is not well in good health ┌╖ PURE MS DOS loaded computer is best place for testing your software if ╘╝ any problems are out. In most cases computers on which your programs will run are filled with different configuration. That means that sometime configuration of computer (hardware, software, TSR, DOS versions) can be causing problems. To see if that's the case i have used BOOTSYS program which allows me boot different configuration of my 386/40 for testing. One from them can be called PURE MS DOS test configuration. It looks like this: CONFIG.SYS FILES=20 BUFFERS=16 DOS=HIGH DEVICE=C:\HIMEM.SYS STACKS=0,0 SHELL=C:\COMMAND.COM /E:768 /P AUTOEXEC.BAT @ECHO OFF PROMPT $P$G \DOSODI\LSL \DOSODI\EXP16 \DOSODI\IPXODI \DOSODI\NETX F: LOGIN NET.CFG FILE HANDLES=100 See, NOTHING TSR loaded, NO device drivers at all. NO SETVER.EXE (anyway why to use it when all must be already DOS 5.0 aware). Just load MS DOS and then DOS ODI network drivers and log into network. NO EMS memory. For more PURE configuration can be HIMEM.SYS and DOS=HIGH removed and network drivers not loaded and application tested from local harddisk. With this configuration one can be sure to eliminate ALL drivers, TSR, additions to your BIOS or VIDEO BIOS and test Clipper application. IF THIS is working and others are not? Look for configuration problems.. ┌╖ What about this SET magic of my Novell Netware 3.11 server, isn't good ╘╝ to try to tune it? Novell Netware 3.11 operating system is selfconfiguring itself against needs from workstations and server. In many cases is good anyway to tune server manually for better performance, or at least take a look what's going on. Following is list of all SET parameters which CAN be related to SPEED of Clipper applications. ■ Maximum Concurrent Disk Cache Writes Increase for WRITING having better performance Decrease for READING having better performance ■ Dirty Disk Cache Delay Time Increase for Small writes having better performance Decrease for better security of data (are sooner written to disk) but with performance going down ■ Dirty Directory Cache Delay Time Increase for better performance and less data security Decrease for better security of data, but slows down ■ Maximum Concurrent Directory Cache Writes Increase for better writing Decrease for better reading ■ Directory Cache Allocation Wait Time Increase will slow down directory accessing time Decrease will FAST directory accessing time ■ Directory Cache Buffer Non Referenced Delay Increase will give faster access to directories Decrease will slow access to directories, but will give more memory for other server tasks ■ Maximum Directory Cache Buffers Increase will give better directory access Decrease when memory in server is required, slows directory access ■ Minimum Directory Cache Buffers Increase will give better initial directory access to server, it's effective only for time of start of server ■ Immediate Purge Of Deleted Files if ON, all deleted files are immediately removed from server, SALVAGE utility has not effect after. Can make server MUCH faster, but make impossible to undelete files ■ Turbo FAR Re-Use Wait Time Increase is good for database files because Turbo-Fat is kept longer in memory. But needs more server memory Decrease saves memory of server ■ Minimum File Delete Wait Time How long file MUST stay (may not be marked ready for Purge). Can help with keeping deleted files. ■ File Delete Wait Time After how long file is marked ready for purge and may be therefore automatically purges from disk. Can help to manage deleted files. ■ Maximum Subdirectory Tree Depth Automatically it's 25 directories level maximum which Netware will allow. If your Clipper application runs with more directories level, increase this number. ■ NCP File Commit When it's ON (default) any application calling File Commit NCP command WILL make HARD flush of data from memory to harddisk of server. Set it to OFF for better performance of server when CLipper application is using too much of DBCOMMIT() calls. ■ Console Display Watchdog Logout Can help you to determine WHY your programs are disconnected from network without visible reason. ■ Maximum Packet Receive Buffer It's limiting number of receive buffers for data from workstations. In case of network error without hardware cause increase this number because server cannot server all your workstation requests. ■ Maximum Record Locks Per Connection Defaults to 500 and means that your program can make maximum of 500 records locked in your databases. If Clipper application is receiving errors of Record locking kind, increase this number. It's PER WORKSTATION, not FOR WHOLE NETWORK. ■ Maximum File Locks Per Connection Default to 250 and means that your program can lock maximum of 250 files (by flock()). It's quite enough because it's also real maximum of files available from network at all. ■ Maximum Record Lock Defaults to 20000 and means maximum of record locks per whole server from all workstation. By default it's space for 40 workstations using full available 500 records lock each. It should be enough, otherwise can be increased. Decreasing can save some server memory and keep better performance. ■ Maximum File Lock Default to 10000 and means maximum of file locks per whole server from all workstations. By default it's space for 40 workstations using full available 250 file locks each. It should be enough, otherwise can be increased. Decreasing can save some server memory and keep performance better. ■ Auto TTS Backout Flag When ON (default) server will automatically after crash backout all active transaction during start. When OFF supervisor must react for server's questions. ■ TTS Abort Dump Flag When ON will create every time TTS$LOG.ERR file in root of volume. This file can log all transactions for analyzing. ■ Maximum Transactions Maximum number of transactions per server from all workstations. Should be enough, but can be changed. ■ Enable Disk Read After Write Verify Can make server as twice fast when it's OFF, but then there will be NO verification of data after writing which can lead in data integrity problems. In disk with own READ/WRITE verify on board it's possible switch this OFF. ■ Maximum outstanding NCP searches When corrupted or invalid directories are encountered in some application is good time to increase this. ■ Allow Unencrypted Passwords Novell 3.11 introduced all passwords as encrypted when travelling through network. If 2.x server with non-update utilities is in the same network as 3.11 server is good to set this to ON. ■ Maximum Service processes Increasing makes better performance Decreasing slows down, but gives more memory for other server tasks. Most from those SET command should be checked once a while in 3.11 server via MONITOR program where are visible most important setting. This check should include check for: Minimum Maximum Sample Reality Service Processes 20 4 Packet Receive Buffers 10 100 10 Directory Cache Buffers 20 500 110 In this sample is good time to start about thinking parameter called Minimum Directory Cache Buffers because it can fast up starting of server and users coming to access their directories. It can be inc- reased to 100 or 110. Other parameters are in order and nothing is near of maximum, therefore it has enough room to grow. ┌╖ When the SPEED is main problem ╘╝ SPEED of Clipper application is main problem always and in network it's one from most important moments. What to do for maximum speed of your program: - make sure that network is enough fast. It can be done by checking transfer speed of your network by simply running SystemInfo from Norton Utilities set. Example numbers: Novell Netware 3.11, Ethernet 8bit card 280~300 KB/sec Novell NEtware 3.11, Ethernet 16bit card 320~370 KB/sec Novell Netware 2.2, Ethernet 16bit card 230~270 KB/sec Lantastic 4.01, Ethernet IDEAL condition 200~250 KB/sec Lantastic 4.01, Ethernet REAL condition 150~200 KB/sec Lantastic older versions 50~100 KB/sec Novell Netware 3.11, Arcnet (2.5MB) 100~170 KB/sec Lan Manager, Ethernet 100~150 KB/sec Lantastic 4.01, 2MB twisted pair 30~ 50 KB/sec Serial network (RS232C) 10~ 30 KB/sec If your number are much lower than those it's time to thinking about: ■ network card is on wrong IRQ and is slowed down ■ network card is faulty and should be replaced ■ network drivers (Shell) is loaded high and therefore slow use standard one (like NETX) and don't load it high ■ network card should be upgraded from 8bit card to full 16bit card when possible ■ computer itself is very slow ■ network card is using DMA or MEMORY technique of communication upgrade to better card using only IO and IRQ ■ server has problems with network card. In server it should be ALWAYS _very_ fast 16bit (minimally) card, otherwise whole network is slow ■ on Novell try to use BNETX for increasing of speed ■ Server is very short on memory and has not enough resources to react to network requests ■ Server is very slow (typical server should be at least 386/25) ■ Server has VERY slow disk or faulty controller or driver ■ There are wiring problem, check number of errors in card statistics or check connection between stations and recheck cabling. Weak connector, missing terminator, interference, not correct card can be reason for slow communication ■ upgrade your network version to something significantly faster and reliable (like Lantastic to Novell Netware, Novell Lite to Lantastic or Novell Lite to real Netware). ■ Upgrade your network topology and type. ARCNET is slower than Ethernet (2.5MB against 10MB) and so on. - Make as much as memory available for your application This is covered somewhere else in this article, therefore go there. If Clipper application has not enough memory it must make more disk swapping and overlaying and therefore is slowing down - Make EMS memory available for your Clipper application Clipper 5.01 can use EMS memory for overlays and data, therefore enough of EMS (2MB at least) can make significant difference in speed - disk overlay is not used and EMS is MUCH faster. - Place temporary files on RAMDISK or HARDDISK instead of network. If your network is not faster than your harddisk, try to point TEMP files to harddisk (or ramdisk). It's faster and also network traffic will be much smaller - When linking application try to move often used modules in ROOT instead of letting overlay them. It can have impact on using of memory, but can make application MUCH faster. When for example is your application using function CHECKIT() every start of MENU on screen, then maybe place it to ROOT and it will not be loaded again and again over. But think about memory usage of course... - place your .EXE file on your local harddisk or RAMDISK and access only data files if it is possible. .EXE will be loaded faster from your disk and also all reading of overlaid part will be faster - make sure that your machine is running at highest possible speed. Most of AT machines have speed switch on chassis or possibility to use CTRL ALT + (big gray plus, not the small one) for switching to high speed. Use some SystemInfo or SPEED utility to see if it's really running fastest as can. - Do not expect that 8088 or 8086 machines will be speed deamons for Clipper application. It will be VERY slow. Average SLOW AT286/12Mhz is still 4 times faster than average 808x machine. Upgrade those machines as fast as possible or use them for other jobs (like print servers they are of great use for matrix printers!). - When upgrading 8088, 8086 or 80286 machines it's good to upgrade them at least to 80386SX/16Mhz with some 2MB or 4MB of memory. - Make sure that your computer has not: ■ slow harddisk which can make lot of troubles ■ slow harddisk controller ■ slow video card. Upgrade all those CGA, MCGA and HERCULES cards to VGA and all will be better in displaying speed ■ slow memory chips requiring additional WAIT STATES ■ keyboard in bad mode because it will work, but slowing down ■ faulty card in system bus because it will slow down all I/O operations - make sure that your program is written as much effective as possible because that's main point of many programs running very slow. Some tips for this: - do not use PRIVATE or PUBLIC - use your indexes as good as possible. When one can make SEEK instead of skipping through database, it's much faster - use DBFSIX replacement of DBFNTX because it's much faster in network and offers much more for optimization - use MACH6 query optimizer for your database operations - EXTENDBASE software for your server should help also - don't generate unneeded files - optimize your databases to not have so much of them without reason - pack sometime .DBT files and .DBF files because if file is too big, it's slower - reports generated from your files which are not ORDER dependent is good to generate with SET ORDER TO 0 because it will save time to access your index and also will reduce network traffic - SUBNTX or DBFSIX is also handy for creating subsets of index files ┌╖ OOPS and what about those terrible problems with indexes ╘╝ Most from us experienced already some 1210 Internal errors or index corruptions. There is NOT most probably bug in Clipper 5.01 way of handling .NTX files, but there WAS bug in 5.0 version causing corruptions in case of BIG applications. Most from index corruption are related to: - index key expression is NOT STABLE. It means it HAS NOT EXACTLY the same size EVERY time it's invoked. TRIM() functions are suicide when used in index key. Remember that index key MUST have always the same length which MUST be the same also when EMPTY parameters are passed to index key (that's a way how Clipper is making initial size for generating a index files). - index key expression is NOT UNIQUE. It means it's RELATED to some other WORK AREAS or FUNCTIONS which can give different results when making and when updating index. Be absolutely sure that there is NO WAY how to make INDEX ON ADRES->NAME+DATA->CODE even when SET RELATION is invoked between ADRES and DATA. No discussion about this, it's even documented in Clipper documentation - Index files are not opened in THE SAME order. It's VERY important that two programs using the same .DBF with the same .NTX (or any other) file is opening them in the same order. What is therefore wrong is for example: App1 SET INDEX TO NAME,LEVEL,CODE App2 SET INDEX TO NAME,CODE,LEVEL In this example most sure LEVEL and CODE are two indexes ready for become corrupted. Why? Because Clipper will update index keys in order they are defined in application and when two Applications will come to LEVEL and CODE it will get updated in different order and can destroy index. - Index file are not opened always when database is updates. It's again VERY important to open ALL index files and have them OPEN at moment database is updates. Don't try to stick on rule that 'when index key related fields are updated', stick on rule that ALWAYS when database is updated. Therefore this is wrong: App1 SET INDEX TO NAME,LEVEL,CODE App2 SET INDEX TO NAME,LEVEL CODE index in this case is subject of destruction which will be manifested by missing keys from index because APP2 will not update correctly this index (because it's not open). - if DBFSIX is your horse for race, then don't forget to update to latest revision (which is 2:30am) and which is fixing few little index related problems finally. A word from author───┐ ─────────────────────┘ Final word. I tried in two days compile as much as knowledge which i was keeping in my head and occasionally just copying out to Compuserve Mail as answers to most common question. I'm sure that i forget something that might be interesting or important, therefore if it's your experience or knowledge that can be added into this Network Guide for Clipper, please feel free to write me or tell me (CIS ID is 100064,2343 and i'm checking mail every working day). I can later release newer versions with updates. Finally for those speaking English as mother tongue. I'm CZECH and i'm working now in The Netherlands (which anyway speaks Dutch) and therefore my English for sure look funny (formulation) and must have many mistakes or misused word. Take a fun i have it also. Anyone wishing make language correction of this text is anyway welcome. And please don't write angry letter or angry comments about something you don't agree with. It can be in many cases. Daniel 100064,2343 Compuserve 2:285/608@FidoNet (31)-10-4157141 FAX to NETCONSULT/DANIEL (31)-10-4157140 VOICE to NETCONSULT/DANIEL